Beowulf – A New Hope for Parallel Computing?

The Beowulf model for clusters of commodity computers[15, 17, 18] has become very popular over the last year, particularly amongst university research groups and other organisations less able to justify large procurements. The Beowulf concept is usually applied[16] to clusters of Personal Computers running Linux, but other platforms and operating systems can also be considered as providing similar functionality [13, 14]. We discuss the criteria for what constitutes a Beowulf cluster, describe our initial experiments with prototype clusters of PCs and iMacs, and review possibilities for construction of a large well-networked system. We have experimented with the configuration and set up of a heterogeneous cluster of PCs on various network fabrics including conventional and fast Ethernet. Design of a scalable network structure is particularly important for a scalable system. We discuss some options for using switch-based infrastructure built from off the shelf switch components. A range of economic as well as technical issues complicate the design of a suitable Beowulf system for a given set of applications. We consider two major requirements scenarios: running a large number of serial applications; and running a small number of highly parallel applications. Of particular interest to us is the situation where individual nodes must be heavily configured with disk and memory. Parallel computing research issues seem particularly relevant to Beowulf clusters, including those of fault tolerance, scheduling and dynamic problem decomposition. We describe our ongoing research activities in these areas and speculate over the impact Beowulf style clusters will have on parallel computing.